Summary:
The sample efficiency challenge in Deep Reinforcement Learning (DRL) compromises its industrial adoption due to the high cost and time demands of real-world training. Virtual environments offer a cost-effective alternative for training DRL agents, but the transfer of learned policies to real setups is hindered by the sim-to-real gap. Achieving zero-shot transfer, where agents perform directly in real environments without additional tuning, is particularly desirable for its efficiency and practical value. This work proposes a novel domain adaptation approach relying on a Style-Identified Cycle Consistent Generative Adversarial Network (StyleID-CycleGAN or SICGAN), an original Cycle Consistent Generative Adversarial Network (CycleGAN) based model. SICGAN translates raw virtual observations into real-synthetic images, creating a hybrid domain for training DRL agents that combines virtual dynamics with real-like visual inputs. Following virtual training, the agent can be directly deployed, bypassing the need for real-world training. The pipeline is validated with two distinct industrial robots in the approaching phase of a pick-and-place operation. In virtual environments agents achieve success rates of 90 to 100%, and real-world deployment confirms robust zero-shot transfer (i.e., without additional training in the physical environment) with accuracies above 95% for most workspace regions. We use augmented reality targets to improve the evaluation process efficiency, and experimentally demonstrate that the agent successfully generalizes to real objects of varying colors and shapes, including LEGO® cubes and a mug. These results establish the proposed pipeline as an efficient, scalable solution to the sim-to-real problem.
Spanish layman's summary:
Presentamos un método para transferir agentes de aprendizaje por refuerzo profundo del entorno virtual al real directamente, mediante una red SICGAN que traduce las observaciones visuales. Se valida en dos brazos robóticos, logrando más del 95 % de precisión con objetos reales.
English layman's summary:
We present a method to achieve a zero-shot transfer DRL agents from virtual to real settings, using a SICGAN network that translate visual inputs. Validated on two robotic arms, it achieves over 95% accuracy with diverse real-world objects.
Keywords: Transfer learning; Deep reinforcement learning; Domain adaptation; Sim-to-real; Zero-shot
JCR-JIF Impact Factor and WoS quartile: 9,000 - Q1 (2025)
DOI reference:
https://doi.org/10.1016/j.engappai.2025.111510
Published on paper: November 2025.
Published on-line: July 2025.
Citation:
L. Güitta-López, L. Güitta-López, J. Boal, A.J. López López, "Sim-to-real transfer via a Style-Identified Cycle Consistent Generative Adversarial Network: Zero-shot deployment on robotic manipulators through visual domain adaptation", Engineering Applications of Artificial Intelligence, Vol. 159, nº. Part A, pp. 111510, November 2025. [Online: July 2025] doi: 10.1016/j.engappai.2025.111510
